由于课程中的训练样本极端不平衡，长尾实例分割是一个具有挑战性的任务。它导致头部课程的严重偏差（含有多数样本）对尾尾。这呈现“如何适当地定义和缓解偏见”最重要的问题之一。先前作品主要使用标签分布或平均分数信息来表示粗粒偏置。在本文中，我们探索挖掘困难的矩阵，该矩阵携带细粒度的错误分类细节，以减轻成对偏置，概括粗液。为此，我们提出了一种新颖的成对类余额（PCB）方法，基于混淆矩阵，在训练期间更新以累积正在进行的预测偏好。 PCB在培训期间生成正规化的纠错软标签。此外，开发了一种迭代学习范例，以支持这种脱结的渐进和平稳的正则化。 PCB可以插入并播放任何现有方法作为补充。 LVIS的实验结果表明，我们的方法在没有钟声和口哨的情况下实现最先进的性能。各种架构的卓越结果表明了泛化能力。

知识蒸馏在分类中取得了巨大的成功，但是，仍然有挑战性。在用于检测的典型图像中，来自不同位置的表示可能对检测目标具有不同的贡献，使蒸馏难以平衡。在本文中，我们提出了一种有条件的蒸馏框架来蒸馏出所需的知识，即关于每个例子的分类和本地化有益的知识。该框架引入了一种可学习的条件解码模块，其将每个目标实例检索为查询的信息。具体而言，我们将条件信息编码为查询并使用教师的表示作为键。查询和键之间的注意用于测量不同特征的贡献，由本地化识别敏感辅助任务指导。广泛的实验表明了我们的方法的功效：我们在各种环境下观察到令人印象深刻的改进。值得注意的是，在1倍计划下，我们将通过37.4至40.7地图（+3.3）与Reset-50骨架的Restinetet提升。代码已在https://github.com/megvii-research/icd上发布。

在本文中，我们提出了一种用于一般物体检测的第一自蒸馏框架，称为LGD（标签引导自蒸馏）。以前的研究依赖于强大的预酝酿教师，以提供在现实世界方案中可能无法使用的指导知识。相反，我们通过对象之间的关系间和帧间关系建模来生成一个有效的知识，只需要学生表示和常规标签。具体而言，我们的框架涉及稀疏的标签外观编码，对象间关系适应和对象内的知识映射，以获得指导知识。他们在培训阶段共同形成隐式教师，动态依赖标签和不断发展的学生表示。 LGD中的模块与学生检测器的端到端训练，并在推理中丢弃。实验上，LGD在各种探测器，数据集和广泛的任务上获得了体面的结果，如实例分段。例如，在MS-Coco DataSet中，LGD将Reset-50下的REDINENT改善2倍单尺度培训，从36.2％到39.0％地图（+ 2.8％）。它在2倍多尺度培训下使用Resnext-101 DCN V2等FCO的探测器增加了更强大的探测器，从46.1％到47.9％（+ 1.8％）。与古典教师的方法FGFI相比，LGD不仅在不需要佩金的教师而且还可以降低固有的学生学习超出51％的培训成本。

While federated learning has shown strong results in optimizing a machine learning model without direct access to the original data, its performance may be hindered by intermittent client availability which slows down the convergence and biases the final learned model. There are significant challenges to achieve both stable and bias-free training under arbitrary client availability. To address these challenges, we propose a framework named Federated Graph-based Sampling (FedGS), to stabilize the global model update and mitigate the long-term bias given arbitrary client availability simultaneously. First, we model the data correlations of clients with a Data-Distribution-Dependency Graph (3DG) that helps keep the sampled clients data apart from each other, which is theoretically shown to improve the approximation to the optimal model update. Second, constrained by the far-distance in data distribution of the sampled clients, we further minimize the variance of the numbers of times that the clients are sampled, to mitigate long-term bias. To validate the effectiveness of FedGS, we conduct experiments on three datasets under a comprehensive set of seven client availability modes. Our experimental results confirm FedGS's advantage in both enabling a fair client-sampling scheme and improving the model performance under arbitrary client availability. Our code is available at \url{https://github.com/WwZzz/FedGS}.

预测药物目标相互作用是药物发现的关键。最近基于深度学习的方法显示出令人鼓舞的表现，但仍有两个挑战：（i）如何明确建模并学习药物与目标之间的局部互动，以更好地预测和解释； （ii）如何从不同分布的新型药物目标对上概括预测性能。在这项工作中，我们提出了Dugban，这是一个深层双线性注意网络（BAN）框架，并适应了域的适应性，以明确学习药物与目标之间的配对局部相互作用，并适应了分布数据外的数据。 Dugban在药物分子图和靶蛋白序列上进行预测的作品，有条件结构域对抗性学习，以使跨不同分布的学习相互作用表示，以更好地对新型药物目标对进行更好的概括。在内域和跨域设置下，在三个基准数据集上进行的实验表明，对于五个最先进的基准，Dugban取得了最佳的总体表现。此外，可视化学习的双线性注意图图提供了可解释的见解，从预测结果中提供了可解释的见解。

Masked image modeling (MIM) performs strongly in pre-training large vision Transformers (ViTs). However, small models that are critical for real-world applications cannot or only marginally benefit from this pre-training approach. In this paper, we explore distillation techniques to transfer the success of large MIM-based pre-trained models to smaller ones. We systematically study different options in the distillation framework, including distilling targets, losses, input, network regularization, sequential distillation, etc, revealing that: 1) Distilling token relations is more effective than CLS token- and feature-based distillation; 2) An intermediate layer of the teacher network as target perform better than that using the last layer when the depth of the student mismatches that of the teacher; 3) Weak regularization is preferred; etc. With these findings, we achieve significant fine-tuning accuracy improvements over the scratch MIM pre-training on ImageNet-1K classification, using all the ViT-Tiny, ViT-Small, and ViT-base models, with +4.2%/+2.4%/+1.4% gains, respectively. Our TinyMIM model of base size achieves 52.2 mIoU in AE20K semantic segmentation, which is +4.1 higher than the MAE baseline. Our TinyMIM model of tiny size achieves 79.6% top-1 accuracy on ImageNet-1K image classification, which sets a new record for small vision models of the same size and computation budget. This strong performance suggests an alternative way for developing small vision Transformer models, that is, by exploring better training methods rather than introducing inductive biases into architectures as in most previous works. Code is available at https://github.com/OliverRensu/TinyMIM.

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

Blind image quality assessment (BIQA) remains challenging due to the diversity of distortion and image content variation, which complicate the distortion patterns crossing different scales and aggravate the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and little research has been done on learning strategies to make the regression model produce better performance. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model's performance.

Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To tackle this issue, we present ongoing work and preliminary findings on the possibility for deep models to transfer knowledge from language to music, by finetuning large language models pre-trained on a massive text corpus on only hundreds of MIDI files of drum performances. We show that by doing so, one of the largest, state-of-the-art models (GPT3) is capable of generating reasonable drum grooves, while models that are not pre-trained (Transformer) shows no such ability beyond naive repetition. Evaluating generated music is a challenging task, more so is evaluating drum grooves with little precedence in literature. Hence, we propose a tailored structural evaluation method and analyze drum grooves produced by GPT3 compared to those played by human professionals, exposing the strengths and weaknesses of such generation by language-to-music transfer. Our findings suggest that language-to-music transfer learning with large language models is viable and promising.